Multi-scale fully convolutional neural networks for histopathology image segmentation: From nuclear aberrations to the global tissue architecture

Histopathologic diagnosis relies on simultaneous integration of information from a broad range scales, ranging nuclear aberrations ($\approx \mathcal{O}(0.1{\mu m})$) through cellular structures \mathcal{O}(10{\mu to the global tissue architecture ($\gtrapprox \mathcal{O}(1{mm})$). To explicitly mimic how human pathologists combine multi-scale information, we introduce family multi-encoder FCNs with deep fusion. We present simple block for merging model paths differing spatial scales in relationship-preserving fashion, which can readily be included standard encoder-decoder networks. Additionally, context classification gate is proposed as an alternative incorporation context. Our experiments were performed three publicly available whole-slide images recent challenges (PAIP 2019, BACH 2020, CAMELYON 2016). The architectures consistently outperformed baseline single-scale U-Nets by large margin. They benefit local well and particularly combination both. If feature maps different are fused, doing so manner preserving relationships was found beneficial. Deep guidance loss appeared improve training at low computational costs. All models had reduced GPU memory footprint compared ensembles individual trained image scales. Additional path fusions shown possible cost, opening up possibilities further, systematic task-specific optimization. findings demonstrate potential presented human-inspired, end-to-end trainable, histopathologic extensive largely